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ABSTRACT 

Thi s paper provides an cvervi ew of a teacher 
evaluation system designed to improve the education of preservice 
teachers at Auburn University (Alabama) by promoting self-assessment, 
reflection, and professional growth. The system uses a portfolio 
approach which includes a simulation exercise in pedagogical decision 
making. The portfolio contains five required college-wide components, 
as well as program-specific components and supplemental components 
determined by individual students. The five college-wide components 
include a professional resume, self-evaluation, most successful 
lesson plan, least successful lesson plan, and student evaluation 
instrument. The simulation exercise uses the Simulation of 
Interactive Decision-Making, in which problem situations are o. f ered 
along with three or more suggested responses and the student 
evaluates the appropriateness of each response. The paper reviews 
methods traditionally used in teacher evaluation; issues a call for 
alternatives such as portfolios and simulation exercises; discusses 
issues in establishing and implementing a portfolio-based system, 
including the perspectives of interns, university supervisors, and 
educational employers on theoretical and practical concerns; 
discusses revisions made to the portfolio process to further 
integrate it within the teacher education curriculum; and provides an 
overview of the development and revision of the simulation component. 
(Contains 51 references.) (JDD) 
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An Evaluation Approach for the Development of Preservice Teachers 

A mandate from the State Department of Education requires that all students in teacher 
preparation pass a comprehensive exit examination on content and professional knowledge prior to 
being recommended for certification. This state mandate applies to all students who entered a teacher 
education program after September 1989. In response to this state mandate, the College of Education 
established an Exit Examination Committee consisting of members from different departments in the 
college. This committee was charged with reviewing the alternatives which comply with the mandate 
and recommending an exit examination plan. Based on feedback from college faculty and existing 
literature, the efforts of the committee have resulted in an evaluation plan which requires each student 
to complete a portfolio and situational simulation exercise. The portfolio phase of this evaluation plan 
was first implemented during the spring quarter of 1993. Since then, the specific requirements this 
portfolio plan have been revised based on quarterly feedback from students and university supervisors. 

This paper will provide an overview of a teacher evaluation system designed to improve the 
education of preservice teachers. The preservice evaluation system described in this paper primarily 
uses a portfolio approach of which one component is a simulation exercise in pedagogical decision- 
making (Shannon. 1994). This paper is divided into three sections discussing the efforts made toward 
the development of a preservice teacher evaluation system that promotes self-assessment, reflection, 
and professional growth. The first section provides an overview of methods traditionally used in 
teacher evaluation and issues a call for alternatives such as portfolios and simulation exercises. The 
second section includes a discussion of the issues faced in establishing and implementing a portfolio- 
based preservice teacher evaluation system. In this section, the perspectives of interns, university 
supervisors, and educational employers on theoretical and practical concerns faced in the development 
of portfolios are represented. This second section also includes a discussion of the specific feedback 
gathered from these sources and the revisions made in the portfolio process to further integrate it 
within the teacher education curriculum. The final section will include an overview of the 
development and revision of the simulation component for inclusion in the overall preservice teacher 
evaluation system. 

» 

Teacher Assessment: An Overview 

Traditional teacher assessment methods 

Traditionally, the assessment of teachers has been conducted using either pencil-and-paper tests 
or observational ratings of classroom performance. State legislatures and boards of education have 
generally relied on pencil-and-paper tests for certification purposes. These tests are intended to 
measure the knowledge that teachers must possess in order to be effective in the classroom. Such 
multiple-choice tests offer certain advantages. They are generally accessible from a variety of test 
publishers, relatively inexpensive, and easy to administer and score. 

Multiple-choice tests, however, offer little in terms of face validity. To convince teachers that 
these tests alone can be used to measure all the knowledge and skills necessary for effective teaching 
is difficult, if not impossible. Whether these tests are valid depends, in part, on whether they achieve 
their purpose of screening out incompetent teachers while advancing competent teachers. This result 
depends, of course, on whether the candidates who are excluded are in fact less competent than those 
who are certified. Evidence to support this kind of validity is difficult to find. 

The National Teacher Examination (NTE) has been used for certification purposes for many 
years (Educational Testing Service, 1940-1976; Wilson, 1985). The procedures used to establish the 
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content validity of the NTE tests for a very long time represented the state of the an. If any teacher 
competency tests possessed content validity, the NTE Common Examinations did. But the clear 
demonstration of content validity is no assurance that the knowledge measured by a test is relevant to 
competent performance in the classroom. 

Several studies of the validity of these tests as predictors of classroom performance have been 
reported over the years; they were reviewed and summarized by Quirk, Witten, & Weinberg (1973). 
These investigators reported median correlations between Weighted Common Examination Total 
(WCET) scores and ratings of teacher performance near zero: 0.05 when student teachers were used 
as subjects, and 0.11 when first-year teachers were used. Evidence of the predictive validity of 
revised National Teacher Examinations has been no more encouraging (Chiu, 1989). 

Since there is no reason to suppose that other, less well constructed multiple-choice tests would 
correlate any higher, it must be concluded that there is no evidence that these conventional tests with 
high content validity are valid predictors of teacher performance. There is no evidence that their use 
in state-wide or national testing programs improves the quality of education. 

In recognition of this melancholy fact, some states have instituted assessment programs in 
which teachers' actual performances on the job are observed and assessed, rather than their 
performance on paper-and-pencil tests. Thus direct observation of teaching offers more in terms of 
credibility for teachers. Furthermore, basing a decision about how effective a teacher is on direct 
observational data is an significant improvement over using multiple-choice tests as the only source 
of information. Observation also allows for teaching to be assessed in the context where it occurs, 
such as first-grade math or tenth-grade biology. 

However, there are several problems with such observational systems which effectively prevent 
them from fulfilling their purpose. One is a problem of timing. Incompetent teachers may teach for 
one or more years and deprive their pupils of the competent instruction to which they are entitled 
. before being screened out of the profession. Therefore, observational systems fall short in achieving 
the purpose of protecting children from incompetent teachers. A second problem is sampling. An 
evaluation of a teacher's ability to teach cannot be made on just one observation. Therefore, many 
observations are needed before an accurate picture of a teacher's "typical M classroom performance can 
be captured. This issue leads to the problem of cost. It costs so much to operate such a program in 
terms of travel and observer expenses that many programs have been discontinued after a few years 
because they were not cost effective. 

Need for alternative teacher assessment methods 

These traditional methods of testing and observation just described have also been criticized 
because they provide a limited view of teaching (Bird, 1990; Darling-Hammond, 1988; Scriven, 1988, 
Shulman & Sykes, 1986). These methods fail to capture the interrelationships of content-area and 
pedagogical knowledge and situational factors such as student diversity and school environment. 
Pencil-and-paper tests are useful in capturing a broad range of a teacher's content area knowledge and 
basic skills, but they fall short in measuring a teacher's ability to teach. 

Classroom observations provide a clearer picture of a teacher's ability to teach, but often they 
fail to capture a teacher's "typical" teaching ability and they are not economically affordable. 
Consequently, the profession has issued a call for the use of multiple or alternative methods 
(Furthwengler, 1986; Haertel, 1991; Shulman, 1988). At the completion of a four-year study 
exploring and developing new methods of teacher evaluation, the Teacher Assessment Project (TAP) 
recommended two approaches. These were: (1) the use of portfolios and (2) simulation exercises 
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(Shulman, 1989). The Auburn University Preservice Evaluation Plan, based on the recommendations 
emerging from the literature and input from current faculty members, primarily uses a portfolio 
approach of which one component of the overall portfolio is a simulation exercise in pedagogical 
decision-making (Shannon, Medley, and Hays, 1993). 

Portfolios 

Portfolios include performance-based elements that allow the teacher to reflect upon the specific 
context in which teaching occurs (Smolen & Newman, 1992; Wolf, 1991a, 1991b). The successful 
development of portfolios requires significant involvement of teachers, both in planning and 
implementation. Teachers help determine what components are best suited for portfolios and establish 
guidelines for their evaluation. The involvement of teachers throughout the development and 
implementation of portfolios helps to establish the overall credibility of the assessment process. 
Portfolios, therefore, possess more face validity than the traditional methods of teacher assessment. 

Portfolios also provide opportunities for professional development. Teachers must reflect upon 
their teaching in order to determine what evidence best supports their ability to teach. This reflection 
affords teachers time to analyze carefully and evaluate their overall teaching ability and thereby 
promote improvement of the overall teaching and learning process. Teachers may collaborate with 
colleagues in discussing their teaching and sharing ideas. This collaboration not only contributes to 
the development of a better portfolio but also to the improvement of teaching. 

The selection of the components which comprise the portfolio is critical, not only in terms of 
whai components are included but also in terms of who selects these components and why each 
component is selected. The determination of components which represent a teacher's best work can 
be made most effectively by that teacher himself or herself. Having teachers choose the components 
to be included in their portfolios encourages more active involvement of teachers in the overall 
assessment process. Supplementing these items with reflection statements further supports the use of 
portfolios as a teacher assessment tool. As teachers write reflection statements, they explain what each 
component is and why it was selected. In addition, the teacher must explain what each component 
represents about his or her teaching ability. This reflection provides the teacher with another 
opportunity to examine his or her abilities and document professional growth. 

Portfolios have been used as a method of teacher evaluation for both inservice and preservice 
teachers. Researchers have discussed issues faced in using portfolios as a method of teacher evaluation 
(Cole, 1992; King, 1991, Wolf, 1991). Other researchers have discussed the use of portfolios within 
teacher education (Barton & Collins, 1993) and, specifically, in the field of science education (Collins, 
1990a, 1990b, 1991). The Ohio Consortium for Portfolio Development (OCPD) (Berry, Kisch, Ryan, 
& Uphoff, 1991) was established at the request of the Teacher Assessment Project (TAP) to address 
specifically issues faced in portfolio development. Consequently, the use of portfolios for national 
certification (Bradley, 1992; NBPTS, 1992), and the integration of portfolios into preservice teacher 
education has also begun to emerge. 

Simulation exercises 

Simulation exercise: provide another alternative to traditional teacher evaluation methods. 
Simulation exercises are designed to assess teaching more realistically than pencil-and-paper tests and 
less expensively than direct observation. Simulations have taken a variety of forms, including in- 
basket tests (Frederickson, Saunders, &. Wand, 1957), computerized planning simulations 
(McNergney, Medley, Allesworth, & Innes, 1983), and interactive teaching decision-making 
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simulations (Shannon, Medley & Hays, 1993a, 1993b). Other simulated exercises include role 
playing, micro-teaching, and other group activities (Haertel, 1991; Jacobson & Stilley, 1992). 
Simulations have also been referred to as Contextual tests/' or M performance tests" (Haertel, 1991). 

Simulation exercises are designed to measure the knowledge of and the actual skill of teachers. 
Those simulations designed to measure teachers' knowledge typically emphasize professional 
knowledge, This professional knowledge is the knowledge that teachers are believed to need in order 
to be effective in the classroom. Traditional tests typically fall short of their intentions to measure 
teachers' professional knowledge. These multiple-choice tests make use of -application" items 
describing an interactive teaching situation and then direct the examinee to select the best option in 
dealing with the described situation. 

A major problem with these traditional types of tests is that they are restricted to measuring 
the academic professional knowledge required to teach. Academic professional knowledge consists 
of knowledge that is useful in answering test items correctly. This type of knowledge must be used 
selectively and when the appropriate situation arises. Unfortunately, this knowledge rarely provides 
for direct solutions to professional problems actually encountered by classroom teachers. The 
professional knowledge that a teacher possesses must be functional within the classroom (Medley, 
Rosenblum, & Vance, 19C9). This functional professional knowledge is the knowledge that a teacher 
draws upon when actually teaching in the classroom, the knowledge which is most useful in the day-to- 
day practice of the teaching profession. 

I suggest that the amount of academic professional knowledge a teacher possesses has much 
less to do with how well a teacher performs in the classroom. How valid a teacher's assessment 
approach is as a predictor of classroom performance depends on the amount of functional professional 
knowledge it measures. I also suggest that the difference between these two kinds of professional 
knowledge reflects how the teacher accesses content rather than the content itself. The ability needed 
to recognize an item of knowledge when it appears as part of a test item is one thing. The ability 
needed to recall the same item of know ledge and apply it to a problem that comes up in the day-to-day 
practice of teaching is another. And, it is this latter ability rather than the former that determines how 
effectively a teacher performs the tasks of teaching. It is this functional professional knowledge that 
a teacher assessment approach must measure if it is to be valid. 

Since paper-and-peneil tests have been designed for and are particularly adapted to the 
measurement of academic knowledge, it is no surprise to anyone that scores on them do not relate to 
the classroom performance of teachers. This generalization is true even when the test consists mostly 
of "application items," items that require the candidate to evaluate suggested solutions to descriptions 
of typical teaching problems. The development of different kinds of assessments, designed to measure 
functional professional knowledge, is necessary. Simulation exercises provide opportunities for 
teachers to examine, via written text or video, actual teaching situations and use their functional 
professional knowledge to analyze each teaching situation. Simulations can be designed to measure 
the functional professional knowledge of teachers more effectively than pencil-and-paper tests. 

Simulations require teachers to perform specific functions of the teaching process such as 
planning, instruction, or evaluation. These functions may include teaching a lesson to a group of 
students or evaluating student work, Teachers are asked to perform these specific teaching functions 
under conditions that resemble the actual teaching context as closely as possible. These conditions are 
usually created at a common assessment center where all teachers engage in the same simulation 
exercise. The advantages gained are those of consistency and cost. That is, these simulated conditions 
are consistent for all teachers and less expensive than directly observing each individual teacher at his 
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or her own classroom. 

The degree to which each simulated exercise captures the reality of the teaching process 
determines the overall validity of the simulation approach. First of all, teachers must perceive the 
simulation as realistic. In other words, support must be offered for face validity. In addition, the 
specific tasks identified in each simulation exercise should be consistent with the actual tasks of 
teaching. This aspect of simulation development requires extensive observation of teaching, job task 
analyses, and consultation with experts in teaching effectiveness to establish the simulation's content 
validity. Last, the results of simulation exercises must reflect actual teaching. How well a teacher 
performs in the actual classroom should be consistent how well he or she performs in the simulation 
exercise. Therefore, if experienced teachers perform more effectively in the classroom than beginning 
teachers, they should also do so under simulated conditions. 

Vie Evolution of a Portfolio-Based Evaluation System 

Issues Faced in Portfolio Development 

The first step in developing this portfolio plan was to establish an exit examination committee 
consisting of members from different departments in the College of Education. This committee, based 
on feedback from college faculty and existing literature, developed a pilot evaluation plan which was 
first implemented in the spring quarter of 1993. The specific nature of this evaluation plan and the 
issues encountered during its development will now be discussed. 

Wolf (1991a) outlined some of the most critical questions faced in portfolio development. These 
questions include: (a) how is a portfolio defined?, (b) what purpose(s) should a portfolio serve?, (c) 
what domains of teaching can be documented in a portfolio?, (d) what components (evidence) should 
be included in a portfolio?, (e) how should these components be selected for the portfolio?, and (f) 
how should portfolios be evaluated? Each of these questions will be addressed in the following 
section* relating relevant literature and the efforts we at Auburn University have made to address each 
question as part of our preservice teacher evaluation program. 

Initial portfolio definition 

One of the first issues facing those involved in the development process is the very nature of 
portfolios. Should a portfolio be a "showcase" of a teacher's best work or should it show the 
"development" of a teacher? Careful consideration of the distinctions and implications of each 
approach led us to the recognition of the inherent developmental nature of portfolios. We believe that 
a teacher's best work can only be accomplished over a long period of time and that good teachers 
continuously grow and refine their skills. Furthermore, a portfolio designed for the teacher who is 
a lifelong learner, a developing person and professional, must be of a developmental nature. The 
portfolio proposed for this project will take the form that Barton and Collins (1993) labeled as 
"dynamic assessment" intended to capture growth and change over time. This approach allows for 
the refinement of portfolio entries and teaching ability based on feedback received throughout the 
process. ^ 



Wolf (1991a) defines a portfolio as "a structured collection of evidence of a teacher's best work 
that is selective, reflective, and collaborative, and demonstrates a teacher's accomplishments over time 
anu across a variety of contexts" (p. 2). In accordance with this notion cf portfolios, the intention of 
Auburn University's College of Education's (AUCOE) preservice portfolio assessment plan is to be 
introduced as students enter teacher education. From this point, each student has a minimum of two 
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years to select (and refine) artifacts (components) to be included in his or her teaching portfolio. 
These artifacts are selected so that they document the progress of the student over his or her teacher 
preparation program. Some components are common to all students, some specified by program 
areas, and some determined by the individual student. 

Portfolios have also been compared to faculty tenure dossiers (Collins, 1990b) in that they 
contain completed work, work in progress, and supporting letters from colleagues. Portfolios have 
also been compared to doctoral dissertations in which the teacher may acknowledge the help of others 
but defends the contents of the portfolio on his/her own (Collins, 1990b; Shulman, 1988). This 
procedure is consistent with our current preservice plan in which students collect components over 
time-partially under the advisement of faculty and cooperating teachers— and must justify each 
component when the final portfolio is reviewed prior to state certification. 

Purpose of the portfolio 

One of the purposes served by the portfolio must be evaluative. However, of greater 
importance to us is the quality of teaching and of teacher reflection. Therefore, a second purpose 
served by this portfolio process is that of improvement. We believe that teaching should be evaluated 
in a manner that provides feedback that can be used for improvement. Portfolios provide this 
opportunity for the professional development of teachers. 

We see the primary benefit of our preservice portfolio as one which will afford interns an 
opportunity to learn more about their teaching and to grow professionally. Therefore, the portfolio 
system is structured so interns are engaged in a continual self-evaluation of and reflection on their 
teaching. The portfolio serves a formative purpose so that meaningful feedback can be given and 
interns may improve their teaching knowledge and skill. 

Domains of the portfolio 

The specific domains included in our preservice teacher portfolio have been derived from the 
content of the current teacher education curriculum and teacher competencies mandated by the state 
(Alabama State Department of Education, 1992). The domains can be broadly divided into content-area 
and pedagogical knowledge. These domains are further divided into areas such as classroom 
management, instruction of material, subject matter knowledge, assessment of student learning, and 
professional development. 

These domains are consistent with those domains specified by Bird (1990) and Collins (1990b, 
1991): (a) instruction, (b) planning and preparation, (c) student and program evaluation, (d) interaction 
with colleagues, and (e) interaction with parents and the community. These domains are also 
congruent with the NBPTS's Vision of Teaching, which outlines five core propositions that 
characterize highly accomplished teachers (National Board for Professional Teaching Standards, 1993). 

Components of a portfolio 
Collins (1990b) differentiates between three types of components (evidence): (a) artifacts, (b) 
reproductions, and (c) reflections. Artifacts consist of materials that teachers normally produce such 
as tests, worksheets, lesson plans, and letters to students and parents. Reproductions include 
videotapes, audiotapes, and pictures of bulletin boards. Reflections are explanations and reasons for 
decisions that teachers make. These reflective statements encourage the teacher to think about the 
teaching process, providing a great opportunity for self-evaluation. The portfolios developed by 
preservice teachers at Auburn include all three types of evidence. The individual interns choose 
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specific types of evidence to be included depending upon their teaching domains and their best 
professional judgements, 

As an indication of our experience and thinking about the types of evidence that comprise an 
appropriate portfolio, we will briefly detail how we chose evidence to be included in our preservice 
portfolios. During the pilot phase of our preservice portfolio project, we solicited feedback from 
interns (student-teachers), university supervisors, and educational employers to determine the most 
appropriate types of evidence. Approximately 20 components were evaluated along a 4-point scale 
as either essential, strongly recommended, optional, or not valuable for inclusion in teaching 
portfolios. Many of these types of evidence were derived from Smolen and Newman (1992). 

We saw it necessary to standardize the components required for the initial portfolio in order 
to meet certain time restrictions and to gather systematic feedback on the overall process. After 
reviewing the feedback was received from the various stakeholders, we made several revisions in the 
overall portfolio structure, adding more flexibility to it. Perhaps the most important modification was 
to incorporate more "reflection" in the portfolios. For example, each teacher candidate now selects 
two lesson plans: one which proved highly successful in its execution and one which proved 
unsuccessful. Along with this selection, the candidate includes reflection statements about what he 
or she believes made the one plan successful and the other unsuccessful. Similar reflections are 
written for other artifacts included in the portfolio. In addition, candidates complete a self-evaluation 
in which they examine themselves in terms of each of eight state-designated competencies for teachers. 
Following this general approach to portfolio development, a common supplement of all types of 
evidence in the portfolio are reflective statements. These statements require the interns to explain the 
component and what it demonstrates about his or her teaching. 

vSelection of components 

Following the recommendation of Wolf (1991b), we have been very explicit about the format 
in which the components (evidence) must be prepared, but more flexible in terms of the actual 
selection of specific evidence. We propose a three-part selection process. While there may be some 
components that are relevant to all teachers, specific subject areas will require specific guidelines. 
In addition, individual interns will also determine what best represents their teaching. 

In our selection process, there are specific components which will be required for all interns. 
Currently, these include a professional resume, evidence of planning ability, evidence of the evaluation 
of student performance, and self-evaluation of teaching. The prescription of specific components or 
modification of the above standard components may be made within specific content areas of the 
teacher education curriculum. Consultation with teacher educators within specific disciplines and other 
relevant consultants helped to determine these components. Each specific area was given a list of 
possible items for inclusion in order to determine which components are most appropriate for teacher 
candidates in their subject areas. Each area was also be responsible for determining the guidelines 
under which these components must be prepared and the criteria for their evaluation. Finally, 
individual interns will also be given the opportunity to select additional components to supplement 
those already prescribed for the portfolio. As we follow this three-part selection process with 
Auburn's preservice portfolio system, we have noticed the evolution of the portfolio from an 
evaluation tool administered by the "outsider 1 * to more of a tool for an individual candidate's own 
reflection and self-evaluation. 

This process allows for greater opportunity to capture teaching in the context in which it occurs 
(or occurred). There is no one way to teach effectively. What makes an effective pre-school teacher 
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does not necessarily make an effective high school science teacher. This approach is very consistent 
with that of the National Board in determining specific standards dependent upon a teacher's 
certification area (National Board for Professional Teaching Standards, 1993). This approach is also 
very consistent with the recommendations made from the Teacher Assessment Project (TAP) (Collins, 
1990b, Shulman, 1988). 

Evaluation of portfolios 

Perhaps the most difficult issue related to using portfolios concerns evaluation. The evaluation 
of portfolios raises such questions as who will evaluate the portfolio?, what criteria will be used?, what 
scoring system will be used?, and how will the reliability and validity be documented? These are not 
easy questions, but they still must be addressed in order for a portfolio assessment system to succeed. 
Our evaluation argument rests primarily in favor of validity. The flexibility built into the portfolio 
structure allows for a more accurate assessment of preservice teachers who are working in different 
teaching contexts. The evaluation of portfolios developed by different teachers from diverse teaching 
contexts becomes very complex. Standardizing the contents of the portfolios for all preservice teachers 
would make this process easier in terms of reliability, but validity would suffer. We agree with Elbow 
(1991 ) that "it makes most sense to put our chips on validity and allow reliability to suffer" (Belanoff 
& Dickson, 1991, p. xiii). 

The establishment of criteria for each component of the portfolio during development is critical. 
These criteria can then be shared with preservice teachers and serve as guidelines for the development 
of their portfolios. Each component, as well as the overall portfolio, is evaluated against the 
established criteria. Failure to meet these criteria results in specific feedback on strengths and 
weaknesses so that the preservice teacher has the opportunity to improve future performance. Saed 
researchers have suggested the use of a follow-up interview to discuss the contents of a teacher's 
portfolio (Collins, 1990b; Shulman, 1988). The purpose of this interview is to question the candidate 
about specific elements of the portfolio and allow him or her another opportunity for reflection and 
self-evaluation of his or her overall teaching ability. This approach resembles that of a doctoral 
dissertation defense. A revision in the current portfolio process is being examined to reflect this 
notion. Consideration is also being given to requiring students to have complete portfolios at the very 
beginning of their internship. The portfolio will then be reviewed by the university supervisor 
followed by a conference with the student. This conference interview, while somewhat structured to 
assure reliability and validity, will be designed to allow interns to expand and elaborate not only on 
the contents of the portfolio but also on their growth experiences. The intern will then have the entire 
internship to refine his or her portfolio components, reflections of his or her teaching, and improve 
his or her teaching skills. 

Feedback and Revisions 

Theoretical Concerns 

Feedback was gathered from interns (n-202), university supervisors (n = ?0), and educational 
employers (n=45) regarding: (a) the purpose of portfolios, (b) the usefulness of portfolios, and (c) 
the components of portfolios. Interns and university supervisors were also asked for feedback on the 
development of portfolios. In addition, employers were asked to indicate the extent to which they 
would find portfolios useful in screening and interviewing teacher candidates. 
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The data indicate agreement among these three sources that portfolios provide evidence of a 
teacher candidate's skills, attitudes, and values that cannot be gathered from a resume alone. Interns, 
however, were more likely to perceive portfolios as a "showcase" of a teacher's work, while 
supervisors and employers were more likely to view portfolios as an opportunity for self-reflection. 
Each group also identified portfolios as: (a) useful in fostering the professional development of 
teachers, and (b) useful in measuring teachers' pedagogical knowledge. Supervisors and employers 
were more likely to find portfolios useful in: (a) clarifying the major responsibilities of teaching, and 
(b) providing the employer with valuable information about a student teacher's potential to become 
an effective teacher. Each group identified several components including: (a) a professional resume, 
(b) lesson plans, and (c) teaching philosophy statement as essential to include in a portfolio. Interns 
were more likely to suggest the inclusion of a classroom management plan and a student discipline 
policy. On the other hand, supervisors were more likely to suggest unit plans and student evaluation 
instruments. 

Practical Concerns 

What difficulties are encountered during the development of a portfolio and what can be done 
to prevent these difficulties? The classroom management Man was identified as the most difficult 
component of the pilot portfolio. The main reasons cited for the difficulty of this component included: 
a) the limited amount of guidance provided, b) the limited attention given to classroom management 
in coursework, and c) the lack of actual classroom experience upon which to base the plan. Concerns 
were also expressed within specific program areas (e.g., early childhood, special education) regarding 
the appropriate format of lesson plans and student evaluation instruments. 

Both interns and university supervisors were in agreement in identifying time as the major 
difficulty in developing the portfolio during the internship. There simply was not enough time during 
this period to develop the portfolio adequately. Other notable difficulties for the interns included the 
lack of preparation for developing and organizing a portfolio. The major suggestions for preventing 
these difficulties were: (a) integrating portfolio development with existing coursework, (b) beginning 
the process be fore internship, and (c) providing examples of the portfolio components. 

College of Education Exit Portfolio 

Initial Exit Portfolio (Spring 1993) 

The exit portfolio plan was first implemented during the spring 1993 quarter. The intention 
was to continue refining the components of the exit evaluation plan over the next year or so. The plan 
that will ultimately be in place for the college will be implemented gradually over this time period. 

The initial exit portfolio requirements were kept to a minimum so that they could be completed 
during the duration of the 10-week internship. The required components for the initial portfolio were 
completed by students participating in their internship during the spring quarter 1993. The initial 
portfolio included five components: 

(a) a professional resume, 

(b) classroom management plan, 

(c) lesson plans, over a period of two weeks 
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(d) a student evaluation instrument, and 

(e) a sample of other teacher-made materials 

Revised Portfolio Requirements (Spring 1994) 

Based on the feedback obtained from interns, supervisors, and educational employers during 
the initial portfolio process and the discussions of the exit examination committee, the portfolio 
requirements were revised. A critical part of this revised portfolio is the degree of reflection and self- 
evaluation required. The intern is required to prepare a statement of reflection for each component 
as well as a self-evaluation of his or her teaching. The current portfolio contains five required 
components: 

(a) a professional resume, 

(b) a self-evaluation of one's teaching ability, 

(c) a lesson plan which proved most successful in its execution, 

(d) a lesson plan which proved least successful, and 

(e) a student evaluation instrument. 



Current Portfolio Requirements (Fall 1994) 

At this point, each student completes the requirements for the portfolio during his or her 10- 
week internship, Further expansion of the portfolio and integration with coursework are planned. The 
expanded portfolio will include components determined by the student's major program area and those 
determined by the individual student. These revised portfolio requirements fall under three different 
sections. Students will be able to assemble the materials required for this expanded portfolio over the 
period of a few years. The degree of reflection required in the current portfolio will be maintained 
in this future portfolio. 

Section A - College-Wide Components 

Some components will be required for all students in the College of Education. These 
components will be kept to a minimum. The current number of college-wide required components is 
five. These components are listed above. These specific portfolio requirements may be revised as 
more feedback is gathered on the overall portfolio process. 

Section B - Program-Specific Components 

Students will also be required to include components specified by their program area. Each 
department must first determine the specific program areas responsible for deciding what components 
best reflect the knowledge and skills appropriate for students in their area. These components will 
then be identified as program-area required components. These components should not duplicate those 
already required by the college in Section A. We ask that each program area identify a minimum of 
one portfolio component and a maximum of five. In addition, the program area must establish the 
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criteria for the evaluation of each component. The specific components determined by the program 
area faculty may be derived from required coursework, field experiences, or other significant 
experiences that students engage in as part of their teacher preparation. We identified specific 
components that may be appropriate for portfolios. These components have been compiled from the 
literature reviewed by the committee as well as the suggestions made by interns and supervisors. A 
sample list of these components was distributed to each program area. 

Section C - Individual Components 

The final category of portfolio components will be determined by individual students. Each 
student will select components that supplement those already specified by the college or program area. 
The list shared with program areas will also be distributed to students. Each component selected by 
the student must be accompanied by a statement of reflection. This statement requires the student to 
explain the component and what it demonstrates about his or her teaching. 

A Chronicle of Change 

The nature of the portfolio-based system at Auburn University has evolved over time. There 
has been increased input from supervisors and classroom teachers in helping to refine the process. 
Perhaps the most major modification was to incorporate more "reflection" in the portfolios. For 
example, the number of lesson plans included by each teacher candidate has been reduced from two 
weeks of plans to just two lessons plans: one which proved most successful in its execution, and one 
which proved least successful. Along with this selection, the candidate will include a reflection 
statement, about what, from the his/her perspective, made the one plan most successful and the other 
least successful. This reflection will also accompany the student evaluation instrument/Candidates 
also complete a self-evaluation in which they examine themselves in terms of state-designated 
competencies for teachers. 

In addition, the candidate's program area will determine a minimum of one artifact (maximum 
of five) to be included in the portfolio, The individual candidate will also select from four to six 
artifacts to be included in his/her portfolio. Each artifact will be accompanied by a statement 
explaining what the artifact is and what it indicates about the person as a professional. The guidelines 
for this revised portfolio will be distributed to students upon entry to teacher education. As can be 
seen in this second phase, the portfolio is evolving from less an evaluation tool administered by the 
"outsider" to more of a tool for an individual candidate's own reflection and self-evaluation. 

The Development of Simulation Exercises 

As described earlier, simulations take on many different forms such as in-basket exercises, 
planning simulations, and interactive simulations. Simulation exercises are designed to focus on 
specific teaching skills. Jackson (1966) distinguished between two aspects of teaching which he called 
preactive and interactive teaching. Prcactive teaching, which occurs when no learners are present, 
includes such activities as diagnosing learning difficulties and planning classroom instruction-activities 
which allow the teacher time to consider alternative courses of action, to reflect on what has occurred 
or may occur in the classroom, to seek help from others, and to consult curriculum guides and other 
reference works. 

M 
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During interactive teaching, which occurs in the classroom in the presence of a class of 
learners, problems arise in such rapid succession that each problem must be handled almost instantly, 
before others arise. There is no time to ponder alternatives, nor is it possible to delay a response to 
a difficult problem while the teacher consults a reference or seeks expert advice. Clearly, the 
knowledge-either of subject matter or of pedagogy-that teachers use and the way that they gain access 
to this knowledge are very different in preactive and interactive teaching. These differences are so 
marked that it is to be expected that very different procedures are necessary for assessing these 
different kinds of professional knowledge. 

An additional phase of teaching has been referred to as posractive (Clark & Peterson, 1986). 
It is during this phase that teachers are engaged in reflective thinking. They process the activities 
which have just occurred in the classroom, engaging in a self-assessment of their teaching, make 
revisions, and begin to plan the next steps. Reflective thinking and self-assessment are very important 
characteristics which a certified teacher should possess. 

We have explored various simulation exercises designed to assess the complex preactive 
interactive, and postactive functions of teaching. These simulation exercises must authentically 
represent the teaching task that are designed to simulate and possess construct validity. That is, these 
simulations must be constructed so that they discriminate between teachers who can perform the task(s) 
effectively and those who can not. One such simulation, the Simulation of Interactive Decision- 
making (SID) was designed to measure pedagogical knowledge functional during interactive teaching 
(Shannon, Medley & Hays, 1993a). SID is currently being refined for use in the Auburn College of 
Education Preservice Teacher Evaluation System. A more detailed discussion of SID is provided 
below. 

Simulation of Interactive Decision-Making (SID) 

SID Item format . The type of item used in SID plays a critical role in determining what the 
test measures. A SID items looks rather like an ordinary "application" test item, because it consists 
of a brief verbal description of a problem situation that a teacher might encounter in the classroom, 
and three or more suggested responses that a teacher might make to each problem. A SID item also 
resembles a test item in that the examinee's task is to evaluate the appropriateness of each response 
suggested. The SID item differs from test items, however, in that one, more than one, or no suggested 
responses to a problem may be appropriate or "correct." Since few of the problems that arise during 
interactive teaching have one and only one response that is appropriate or correct, SID items more 
accurately reflect the nature and timing of teacher decision making. 

Mode of administration . There are important differences, also, in the way in which SID items 
are administered. Instead of presenting a set of problems and suggested responses in a printed booklet 
and providing a reasonable amount during which examinees are free to allot as much or as little time 
to each problem as they see fit, SID problems are presented one at a time, and the suggested responses 
to each problem are also presented and evaluated one at a time. The examinee is given one and only 
one opportunity to evaluate each suggested response, and allowed just five seconds in which to do so, 
before the next response is suggested, 

Rationale . The task defined by the simulation is intended to resemble the interactive teaching 
task much more closely than the task defined by a multiple-choice test. The examinee is required to 
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gain access to professional knowledge in a way much more like the way in which a teacher operates 
in the classroom. It is hypothesized that the inore competent a teacher is to apply professional 
knowledge in the classroom, the better the teacher will succeed in applying the same knowledge in the 
simulation. 

Validity of SID . The specifications for SID were established in alignment with indicators of 
teaching effectiveness summarized for the Virginia Beginning Teacher Assistance Program (BTAP) 
(BER, 1984; Brophy & Good, 1986; Crawford & Robinson, 1983; Florida State Department of 
Educatioiv/1980; Smith, 1983; Weber & Roff, 1983). The specific problems presented in SID were 
also examined by a panel of experts which included teachers, school administrators and college faculty. 
In addition, in support of SID's face validity, 97% of the teachers who participated in SID indicated 
that the problems presented in SID2 often or always reflected the types of problems they have 
encountered in the classroom. 

The gathering of this evidence in support of construct validity has been the primary emphasis 
in the earlier phases of SID's development (Hays, 1988; Shannon, 1990). The findings from both 
these studies, reported elsewhere (Shannon, Medley, & Hays, 1993a), have consistently supported the 
construct validity of SID. That is, the results of SID have reflected differences in terms of the training 
and teaching experience. Specifically, those individuals with more training (education) in effective 
teaching methods perform more successfully than those with less or no training in effective teaching 
methods. Also, those with more teaching experience have outperformed those with less or no teaching 
experience. 

Current Development of SID at Auburn University 

The use of the SID as an examination of the professional knowledge of graduating preservice 
teachers is currently being investigated. Efforts made in the development of content for this 
examination and the integration of computer technology are discussed in the following sections. 

Content Development . The content for this examination has been drawn from several sources. 
First, the state of Alabama Beginning Teacher Competencies (Alabama State Department of Education, 
1992) were consulted to provide a overview of specific behavioral expectations for first year teachers. 
These competencies were initially based on research literature of effective teaching. The areas of 
competence mandated by the state help to serve as a framework in which to develop specific 
examination items, or scenarios. Additional literature regarding focusing on knowledge and 
expectations for beginning teachers was also consulted as a basis for scenario development (Reynolds, 
1992; Veenman, 1984, Weinstein, 1988). 

Information on perceived problems of beginning teachers and effective teaching behaviors has 
been gathered from samples of teachers, principals, teacher educators, and preservice teachers across 
the state. This information was used to develop scenarios that capture realistic teaching situations that 
can be integrated into an assessment of graduating preservice teachers. An additional source being 
used to generate scenarios is that of "cases" written by both teachers and university faculty members 
(Greenwood & Parkay, 1989; Kowalski, Weaver, & Henson, 1990; Shulman & Colbert, 1988; 
Shulman & Colbert, 1987; Silverman, Welty, & Lyon, 1992). 
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Computerization . SID is also being modified so that it may be administered using a 
microcomputer. A computer administrative mode will maintain the components of 5 second timing 
intervals between responses and scoring of multiple "appropriate" answers. An important addition will 
be that of a built in scoring program allowing for more immediate feedback to the examinee. 

The simulation will be presented to preservice teachers using an IBM multi-media computer 
with the Classroom Presentation Option (CPO). A Rainbow HD700 viewplate is used in conjunction 
with an overhead projector to project the simulation items so that a room of students may be assessed 
at one time. The primary software necessary for this presentation includes: Windows, ToolBooK, and 
Excel. 

The Classroom Presentation Option is one that allows for the testing of multiple examinees 
from remote student response keypads. Each examinee has a separate remote keypad that is connected 
to a switching station, which is connected to the IBM multi-media computer. As a simulation test item 
is presented to the examinees and a response is called for, a red light is turned on at each remote 
keypad for five seconds to allow for a response. The examinee must simply press either "YES" or 
"NO" during this time period to indicate whether they feel the response is "appropriate" or "not 
appropriate" for the problem described in the scenario. 

All keypads have an identification number so that all responses from each keypad may be kept 
separate. These responses are automatically assembled in an Excel spreadsheet file. A scoring 
program is then used to score each examinee's responses. It is then possible to provide feedback to 
each examinee immediately after the examination is completed. This feedback may be broken down 
by item, or area of competence. 

Future Plans for SID . The refinement of the processes and development of additional scenarios 
that reflect teaching will continue. The successful planning and production of useful scenarios will 
require much assistance from preservice teachers, experienced teachers, and teacher educators. 
Extensive observation of teachers and the preparation of videotaped segments of teaching behavior wili 
be necessary before video can be integrated into the Simulation of Interactive Decision-making (SID). 
The production of these videotaped segments will require the editing of videotapes from classroom 
observations. These video segments will be produced so that they capture teaching situations that 
require the use of the knowledge and skills identified as necessary for beginning teachers. 

Once these teaching segments were complete, they would be further examined in terms of 
validity and applicability to actual classroom practice before being integrated with the computerized 
SID process. Samples of teachers and principals from participating professional development systems 
(PDS), teacher educators, and preservice teachers would be asked to evaluate these video teaching 
segments in terms of their ability to capture a realistic classroom situation that calls for the use of 
professional knowledge. The final videotaped segments could then be integrated with the computerized 
SID. 

Summary 

In this paper, I have discussed the advantages and disadvantages of traditional methods used 
to assess teachers. A call for alternative assessment methods, primarily portfolios and simulations, 
was also made. The preservive evaluation plan underway at Auburn University incorporates both 
these approaches. Issues faced in the development and refinement of a portfolio evaluation system and 
feedback gathered from participants were discussed. Finally, the development of a simulation of 
interactive decision-making was described, 
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